Image processing apparatus and non-transitory computer readable medium

ABSTRACT

An image processing apparatus includes a processor configured to adjust a reproduction parameter for correcting image visibility by changing a degree by which the reproduction parameter is to be changed from a reference value in accordance with a ratio of a second color gamut to a first color gamut. The first color gamut indicates a range in which an image is expressible by a first device. The second color gamut indicates a range in which an image is expressible by a second device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2020-210536 filed Dec. 18, 2020.

BACKGROUND (i) Technical Field

The present disclosure relates to image processing apparatuses andnon-transitory computer readable media.

(ii) Related Art

Japanese Unexamined Patent Application Publication No. 2008-271316discloses an image processing method including a step of calculating anaverage value of pixel values within an image, a step of calculating amaximum value of the pixel values within the image, a step ofcalculating a minimum value of the pixel values within the image, and areplacing step. In the replacing step, pixels having pixel values in arange between a value obtained by subtracting the average value from themaximum value and the minimum value have their pixel values within theimage replaced by using a gradation conversion characteristic in whichthe brightness increases relative to that of pixels outside the range.

Japanese Patent No. 5247910 discloses an image display apparatus thatgenerates output image data by correcting an input image based on theRetinex theory. This image display apparatus includes a separator thatseparates the input image into an illumination light component and areflection light component by using a filter, a corrected illuminationlight component calculator that calculates a corrected illuminationlight component by adjusting the separated illumination light componentusing an illumination light component adjustment parameter for adjustingthe brightness of the separated illumination light component, and agenerator that generates the output image data based on the correctedillumination light component. The corrected illumination light componentcalculator calculates the corrected illumination light component setbased on the illumination light component adjustment parametercorresponding to a degree by which the illumination light componentseparated by the separator and an adjusted illumination light componentobtained by adjusting the level of emphasis of a low gradation componentof the illumination light component are to be mixed and added.

Japanese Patent No. 6357881 discloses an image processing apparatus thatincludes a characteristic information acquiring unit, an emphasis-degreeinformation generator, and a brightness reproduction image generator.The characteristic information acquiring unit acquires characteristicinformation indicating the characteristics related to at least one ofthe distribution of specific lightness areas in a brightness image andthe size of a dark flat area. The emphasis-degree information generatorgenerates emphasis-degree information, indicating the degree of emphasisof a specific component that has an effect on the image quality of thebrightness image, based on the characteristic information. Thebrightness reproduction image generator generates a brightnessreproduction image reproduced such that the specific component of thebrightness image is emphasized based on the degree of emphasis indicatedby the emphasis-degree information. The characteristic informationacquiring unit acquires aggregation-degree information indicating thedegree of aggregation of lightness areas having multiple lightnesslevels in the brightness image as the characteristic informationindicating the characteristics related to the distribution of thespecific lightness areas in the brightness image. The emphasis-degreeinformation generator generates the emphasis-degree information based onthe aggregation-degree information and the weight of each of themultiple lightness levels.

SUMMARY

For example, even if a reproduction parameter to be applied to an imageoutput from a first device, such as a liquid crystal display, is to bedirectly applied to an image output from a second device, such as aprinter, the outputtable color gamut varies between the devices. Thus,the reproduction parameter to be used in the first device may sometimesbe not an appropriate reproduction parameter to be used in the seconddevice.

Aspects of non-limiting embodiments of the present disclosure relate toan image processing apparatus and a non-transitory computer readablemedium that enable improved visibility of an image to be output from asecond device, as compared with a case where a reproduction parameter tobe applied to an image output from a first device is directly applied tothe second device.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided animage processing apparatus including a processor configured to adjust areproduction parameter for correcting image visibility by changing adegree by which the reproduction parameter is to be changed from areference value in accordance with a ratio of a second color gamut to afirst color gamut. The first color gamut indicates a range in which animage is expressible by a first device. The second color gamut indicatesa range in which an image is expressible by a second device.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present disclosure will be described indetail based on the following figures, wherein:

FIG. 1 schematically illustrates the configuration of an imageprocessing system according to an exemplary embodiment of thedisclosure;

FIG. 2 is a block diagram schematically illustrating an image processingapparatus according to a first exemplary embodiment of the disclosure;

FIG. 3 is a block diagram schematically illustrating an image formingapparatus according to the first exemplary embodiment of the disclosure;

FIG. 4 is a block diagram illustrating an example of a functionalconfiguration of the image processing apparatus according to the firstexemplary embodiment of the disclosure;

FIG. 5 illustrates an example of a scenery image for explaining darkareas according to the first exemplary embodiment of the disclosure;

FIG. 6 illustrates another example of a scenery image for explainingdark areas according to the first exemplary embodiment of thedisclosure;

FIG. 7 illustrates an example of a coefficient associated with apercentage of the number of pixels in a dark area according to the firstexemplary embodiment of the disclosure;

FIG. 8 illustrates an example of color gamuts of an input device and anoutput device on an a*b* plane;

FIG. 9 illustrates an example of the color gamuts of the input deviceand the output device on an L*a* plane;

FIG. 10 illustrates an example of a function in which a compression rateand a reproduction-parameter reduction degree are associated with eachother;

FIG. 11 is a flowchart illustrating an example of operation performed bythe image processing apparatus according to the first exemplaryembodiment of the disclosure;

FIG. 12 illustrates an example of a coefficient associated with apercentage of the number of pixels in a dark area when the coefficientaccording to a second exemplary embodiment of the disclosure is a linearfunction;

FIG. 13 illustrates an example of a coefficient associated with apercentage of the number of pixels in the dark area when the coefficientaccording to the second exemplary embodiment of the disclosure is amultidimensional function;

FIG. 14 illustrates an example where the lightness at which windowsalone are extracted as low brightness areas from a scenery image usedfor explaining dark areas is defined as a lightness hierarchical level,and an image is binarized at this lightness hierarchical level;

FIG. 15 illustrates an example where the lightness at which windows andtrees are extracted as low-to-intermediate brightness areas from thescenery image used for explaining dark areas is defined as a lightnesshierarchical level, and an image is binarized at this lightnesshierarchical level; and

FIG. 16 is a block diagram illustrating an example of a functionalconfiguration of an image processing apparatus according to a thirdexemplary embodiment of the disclosure.

DETAILED DESCRIPTION Background of Disclosure

With the current popularization of information and communicationtechnology (ICT) devices, such as smartphones and tablets, anyone iscapable of easily capturing an image at any place. While enhancedconvenience is achieved in this manner, a special emphasis is placed onenhanced visibility of images affected by illumination, such asbacklight and shadows, due to diverse usage environments.

In the field of image processing, brightness-related or color-relatedcorrection and emphasis, such as gamma correction, saturation emphasis,band emphasis, contrast emphasis, and dynamic range correction, areperformed for the purpose of desired reproducibility and improvedvisibility.

Normally, the aforementioned correction and emphasis processes to beperformed include a type of process involving designating the mode andintensity on the user interface (UI) of the software, a type of processinvolving mechanically setting an optimal reproduction parameter withinthe software, and a type of process involving a combination of the twoaforementioned types of processes. In order to mechanically set anoptimal reproduction parameter, an evaluation indicator indicating thequality of reproducibility is necessary. In many cases, a feature valueof an image calculated from an image histogram or frequency is used asthe indicator, and the correction is performed in accordance with thefeature value.

In contrast, in the following exemplary embodiments, an optimalreproduction parameter is set in accordance with a feature value in viewof the color gamut of an output device and the visual characteristics,so as to improve the visibility of dark areas.

First Exemplary Embodiment

A first exemplary embodiment of the present disclosure will be describedbelow with reference to the drawings. In the drawings, identical orequivalent components and sections are given the same reference signs.Furthermore, the dimensional ratios in the drawings are exaggerated forthe sake of convenience and may sometimes differ from the actual ratios.

FIG. 1 schematically illustrates the configuration of an imageprocessing system 10 according to the first exemplary embodiment.

As shown in FIG. 1, in the image processing system 10, image processingapparatuses 100 and an image forming apparatus 200 are connected to acommunication unit, such as a network. As will be described later, thecommunication unit used may be any of various types of networks, such asthe Internet, an intranet, and Ethernet (registered trademark). Althoughtwo image processing apparatuses 100 and a single image formingapparatus 200 are shown in FIG. 1, the number of image processingapparatuses 100 and the number of image forming apparatuses 200 are notlimited. In the first exemplary embodiment, each image processingapparatus 100 is a personal computer (PC). The image forming apparatus200 has various types of functions, such as a printing function. Inaddition to the printing function, the image forming apparatus 200 mayhave a copying function, a facsimile function, and a scanning function.

FIG. 2 is a block diagram illustrating a hardware configuration of eachimage processing apparatus 100 according to the first exemplaryembodiment.

As shown in FIG. 2, the image processing apparatus 100 has a centralprocessing unit (CPU) 101 as an example of a processor, a read onlymemory (ROM) 102, a random access memory (RAM) 103, a storage unit 104,an input unit 105, a display unit 106, and a communication unit 107.These components are connected to one another in a communicable mannervia a bus 108.

The CPU 101 executes various types of programs and controls eachcomponent. Specifically, the CPU 101 reads a program from the ROM 102 orthe storage unit 104 and executes the program by using the RAM 103 as awork area. The CPU 101 controls the aforementioned components andperforms various types of arithmetic processes in accordance with theprogram stored in the ROM 102 or the storage unit 104. In the firstexemplary embodiment, the ROM 102 or the storage unit 104 storesprograms therein.

The ROM 102 stores various types of programs and various types of datatherein. The RAM 103 serves as a work area and temporarily stores aprogram or data therein. The storage unit 104 is a hard disk drive (HDD)or a solid state drive (SSD), and stores various types of programs,including an operating system, and various types of data.

The input unit 105 includes a pointing device, such as a mouse, and akeyboard, and is used for inputting various types of information.

The display unit 106 is an example of a first device and is, forexample, a liquid crystal display. The display unit 106 displays varioustypes of information based on control by the CPU 101. Moreover, thedisplay unit 106 may be of a touchscreen type to function as the inputunit 105. In this description, a liquid crystal display is described asan input device.

The communication unit 107 is provided for communicating with anotherapparatus, such as the image forming apparatus 200 or a server apparatus(not shown), and uses a standard, such as a public line, the Internet,an intranet, Ethernet (registered trademark), fiber distributed datainterface (FDDI), or Wi-Fi (registered trademark).

FIG. 3 is a block diagram illustrating a hardware configuration of theimage forming apparatus 200 according to the first exemplary embodiment.

As shown in FIG. 3, the image forming apparatus 200 has a CPU 201 as anexample of a processor, a ROM 202, a RAM 203, a storage unit 204, aninput unit 205, a display unit 206, a document reader 207, an imageforming unit 208, and a communication unit 209. These components areconnected to one another in a communicable manner via a bus 210.

The CPU 201 executes various types of programs and controls eachcomponent. Specifically, the CPU 201 reads a program from the ROM 202 orthe storage unit 204 and executes the program by using the RAM 203 as awork area. The CPU 201 controls the aforementioned components andperforms various types of arithmetic processes in accordance with theprogram stored in the ROM 202 or the storage unit 204. In the firstexemplary embodiment, the ROM 202 or the storage unit 204 storesprograms therein.

The ROM 202 stores various types of programs and various types of datatherein. The RAM 203 serves as a work area and temporarily stores aprogram or data therein. The storage unit 204 is a hard disk drive (HDD)or a solid state drive (SSD), and stores various types of programs,including an operating system, and various types of data.

The input unit 205 includes a pointing device, such as a mouse, and akeyboard, and is used for inputting various types of information.

The display unit 206 is, for example, a liquid crystal display. Thedisplay unit 206 displays various types of information based on controlby the CPU 201. Moreover, the display unit 206 may be of a touchscreentype to function as the input unit 205.

The document reader 207 takes in documents placed on a sheet feed trayof an automatic feeder (not shown) provided at an upper section of theimage forming apparatus 200 in a one-by-one fashion and optically readseach document so as to obtain image information. Alternatively, thedocument reader 207 may obtain image information by optically reading adocument placed on a base, such as a platen glass member.

The image forming unit 208 is an example of a second device and isconfigured to form, that is, print, an image onto a recording medium,such as paper, based on image information obtained from the imageprocessing apparatus 100 or image information obtained as a result of areading process performed by the document reader 207. In thisdescription, an image forming unit is described as an output device.

The communication unit 209 is provided for communicating with anotherapparatus, such as an image processing apparatus 100 or the serverapparatus (not shown), and uses a standard, such as a public line, theInternet, an intranet, Ethernet (registered trademark), FDDI, or Wi-Fi(registered trademark).

Furthermore, the communication unit 209 connects the image formingapparatus 200 to a public line and exchanges the image informationobtained as a result of the reading process performed by the documentreader 207 with another image forming apparatus having a facsimile (FAX)function.

Each image processing apparatus 100 uses the aforementioned hardwareresources to realize the various types of functions. A functionalconfiguration realized by each image processing apparatus 100 will nowbe described.

FIG. 4 is a block diagram illustrating an example of the functionalconfiguration of the image processing apparatus 100.

Color Space Converter 11

A color space converter 11 converts an input-device RGB image displayedin, for example, sRGB on a display into a lightness/chromaticity colorspace, such as a CIE L*a*b* color space. The components of CIE L*, a*,b* respectively indicate lightness, chromaticity in the red-greendirection, and chromaticity in the blue-yellow direction. In the firstexemplary embodiment, a* and b* will be defined as chromaticitycomponents, and L* will be defined as a brightness component. The colorspace may be of any type so long as the color space is separable intolightness and chromaticity components, such as Lαβ, CAM02, and HSV.

Visibility Reproducer 14

A visibility reproducer 14 performs a reproduction process on abrightness image for enhancing the visibility thereof, so as to generatea brightness reproduction image. In the first exemplary embodiment, thisreproduction process uses the principle of Retinex theory. Areproduction process using the principle of Retinex theory will bedescribed by using a reproduction expression indicated as Expression 1below. In this description, a hat symbol is indicated directly above acharacter in each expression, but is added behind the character in text.

Î(x, y)=αI _(R)(x, y)+(1−α)I(x, y)   Expression 1

With reference to the reproduction expression of Expression 1 as anexample, I(x, y) denotes a pixel value of an original image, and αdenotes a reproduction parameter for emphasis. Furthermore, I_(R)(x, y)denotes a pixel value of an estimated reflectance component, andI{circumflex over ( )}(x, y) denotes a pixel value of an image after thereproduction process. Thus, I{circumflex over ( )}(x, y) denotes a pixelvalue of the original image when α=0, and denotes a pixel value of areflectance image when α=1. Although there are several Retinexreproduction expressions proposed, the reproduction expression used isnot limited to Expression 1 and may be of any type so long as theexpression varies the weight of the reflectance component. Moreover, (x,y) denotes the position of an image.

Assuming that a pixel value with respect to the position (x, y) of abrightness image is defined as I(x, y), I(x, y) is separated into anillumination component L(x, y) and a reflectance component R(x, y) inthe Retinex model, as indicated below.

I(x, y)=R(x, y(L(x, y)   Expression 2

As it is clear from Expression 2, separating one value into two valuesis normally called an ill-posed problem. In an ill-posed problem, theillumination component or the reflectance component is not determinableunless the other component is estimated using some type of method.

The visibility reproducer 14 estimates L(x, y) based on I(x, y). As onecharacteristic of visual perception, a perceptual amount of a singlespot (i.e., a single pixel in an image) of light incident on the retinais known to have an effect on the average ambient brightness of thesingle spot. An ambient brightness corresponds to an estimatedillumination component, and an illumination estimation model is aconvolution of the following function.

$\begin{matrix}{{G( {x,y} )} = {k\;{\exp( {- \frac{x^{2} + y^{2}}{\sigma^{2}}} )}}} & {{Expression}\mspace{14mu} 3}\end{matrix}$

In this case, k denotes a coefficient to be used for normalization suchthat 1 is obtained when integration is implemented for pixels having afilter size for image processing, and a denotes a smoothing scale. Aconvolution image with intense blurriness is obtained as a increases.Expression 3 indicated above is an example, and may be of any type solong as the expression serves as a filter that obtains a similar(smoothing) result. For example, a bilateral filter known as a smoothingfilter that performs edge preservation may be used by modifyingExpression 2.

According to the principle of Retinex theory, it is known that thevisual characteristics of humans estimate illumination light from aroundan area of interest. Although a smoothed image expresses the estimatedillumination light, a suitable scale varies depending on the scene.Therefore, for example, illumination light is desirably estimated byperforming a weighted averaging process from scale 1 to scale N of amultilayer image, as in Expression 4 indicated below.

$\begin{matrix}{{L( {x,y} )} = {\sum\limits_{n = 1}^{N}{W_{n}{{G_{n}( {x,y} )} \otimes {I( {x,y} )}}}}} & {{Expression}\mspace{14mu} 4}\end{matrix}$

In this case, L(x, y) denotes a pixel in an estimated illumination lightimage, G_(n)(x, y) denotes Expression 1 at scale n, I(x, y) denotes apixel in an image, W_(n) denotes a weight of scale n, and a symbolhaving “x” surrounded by a circle denotes a convolution. Specifically,W_(n) may simply be 1/N, or may be variable in accordance with thelayer. Furthermore, L(x, y) calculated in this manner is defined as anillumination component image.

According to Expression 2, a reflectance R(x, y) may be calculated, asin Expression 5 indicated below, based on the estimated illuminationcomponent.

$\begin{matrix}{{R( {x,y} )} = \frac{I( {x,y} )}{L( {x,y} )}} & {{Expression}\mspace{14mu} 5}\end{matrix}$

Specifically, a reflectance R(x, y) is calculated by using a pixel valueI(x, y) of a brightness image and a pixel value L(x, y) of anillumination component image. An image having this reflectance R(x, y)as a pixel value is a reflectance image.

Furthermore, as indicated in Expression 1, a visibility reproductionbrightness image is generated in accordance with the reflectance R(x, y)and the brightness image I(x, y) in Expression 5, and the reproductionparameter α.

Visibility is greatly affected by the reproduction parameter α.Therefore, in the first exemplary embodiment, a reproduction parameteranalyzer 12 is used to estimate a reproduction parameter in aninput-device space from a feature value of an image, and anoutput-device optimal-reproduction-parameter calculator 13 is used toset an optimal reproduction parameter in view of the color gamut of anoutput device.

Reproduction Parameter Analyzer 12

The reproduction parameter analyzer 12 estimates a reproductionparameter from a feature value of an image in the input-device space. Anexample will be described here by using scenery images in FIGS. 5 and 6.In FIG. 5, the trees and the windows in the building are dark areas withpoor visibility. In FIG. 6, the trees and the building are dark areaswith poor visibility. Although the brightness of the trees is the samebetween FIGS. 5 and 6, the brightness of the building is lower in FIG.6, resulting in the overall image in FIG. 6 having poorer visibility. Itis thus determined that the image in FIG. 6 is to be increased inintensity with respect to the reproduction parameter.

As mentioned in the above example, if dark areas have the samebrightness, the larger dark area has poorer visibility. Thus, twofeature values of an image, namely, the percentage of the number ofpixels in the dark area (i.e., the size of the dark area) relative tothe number of pixels in the entire image and the average brightness ofthe dark area, are used. This will be described by using a brightnesscomponent. A brightness image converted from an input-device RGB imagewill be defined as L*(x, y). Then, an image obtained as a result ofbinarizing L*(x, y) at a maximum of 1 and a minimum of 0 based on acertain threshold value is defined as L*′(x, y).

A percentage r, serving as a feature value, of the number of pixels inthe dark area may be expressed as in Expression 6 below, assuming thatthe overall number of pixels in the image is defined as N′ and thenumber of pixels in the dark area is defined as N.

$\begin{matrix}{r = \frac{N}{N^{\prime}}} & {{Expression}\mspace{14mu} 6}\end{matrix}$

Furthermore, an average brightness L*ave of the dark area may beexpressed as in Expression 7 below by using L*(x, y) and L*′(x, y).

$\begin{matrix}{{L*{ave}} = \frac{\sum( {L*( {x,y} ) \times L^{\prime}*( {x,y} )} )}{N}} & {{Expression}\mspace{14mu} 7}\end{matrix}$

Although the reproduction parameter a is calculated by using the featurevalues calculated in Expression 6 and Expression 7, the followingdescription relates to an example where the reproduction parameter a iscalculated based on the average brightness of the dark area.

First, in a case where the reproduction parameter is to be estimated byusing the average brightness of the dark area alone, the reproductionparameter may be expressed as in Expression 8 indicated below.

α=a×L* _(ave)   Expression 8

In this case, a denotes a coefficient set such that a reproductionparameter set based on, for example, a sensory evaluation test executedin advance has a minimal error in accordance with the least squaresmethod.

Output-Device Optimal-Reproduction-Parameter Calculator 13

The output-device optimal-reproduction-parameter calculator 13 sets anoutput-device optimal reproduction parameter by using a compressionrate, saved in advance as data, of the color gamuts of the input device(i.e., the liquid crystal display) and the output device (i.e., theimage forming unit 208).

Normally, when the input device and the output device have differentcolor gamuts, a color not reproducible in the output device has to beconverted into a reproducible color. An example will be described withreference to FIGS. 8 and 9. FIG. 8 illustrates the color gamuts of theinput device and the output device by cutting out an a*b* plane. FIG. 9illustrates the color gamuts of the input device and the output deviceby cutting out an L*a* plane. Because the input device has a largercolor gamut than that of the output device, the colors in the inputdevice have to be converted into colors reproducible in the outputdevice, as in point A to point A′ and point B to point B′. Examples ofthe conversion method include minimizing “color difference” serving as aunit of measurement obtained by converting a color difference into anumeral, and a method of minimizing the color difference whilemaintaining hue.

The color gamut of each of these devices is normally a group of multiplecolors. If the device is a liquid crystal display (i.e., the displayunit 106), the colors displayed on the liquid crystal display aredirectly measured. If the device is a printer (i.e., the image formingunit 208), a patch sample output to, for example, paper is measured. Thecolor gamut of each device may be defined in this manner. With regard tothe colors to be measured, data set based on an international standard,such as ISO, is often used.

The ratio between the sizes of the color gamuts of the two devices, thatis, the ratio between the volumes of the two devices in the L*a*b*space, is expressed as a compression rate. For example, the volume ofthe color gamut (second color gamut) of the output device is comparedwith the volume of the color gamut (first color gamut) of the inputdevice in FIGS. 8 and 9. If the volume ratio is 0.5, the compressionrate is set to 50%. Then, this compression rate is used to set anoptimal reproduction parameter.

In detail, because the input device has the larger color gamut, acolored area is reproduced vividly, whereas a dark area is reproduceddarkly. Therefore, when a reproduction parameter is calculated using thecolor gamut of the input device, the reproduction parameter is largerthan a reproduction parameter calculated using the smaller color gamutof the output device, and is too intense in the output-device space.Thus, based on a relationship shown in FIG. 10, a coefficient K servingas a reproduction-parameter reduction degree by which the reproductionparameter when the compression rate is 50% is changed from a referencevalue (1.0) in a case where the color gamut of the input device and thecolor gamut of the output device are equal to each other is calculated.An output-device optimal reproduction parameter α′ may be expressed asin Expression 9 by using the coefficient K and the reproductionparameter α calculated by the reproduction parameter analyzer 12.

α′=K×α  Expression 9

Finally, the visibility reproducer 14 calculates a in Expression 1 as α′in Expression 9, thereby generating a visibility reproduction brightnessimage in view of the color gamut of the output device.

Although the relationship between the compression rate and thereproduction-parameter reduction degree is indicated as a linearrelationship, any function shape may be used, including a nonlinearcurve, so long as the value of the reproduction-parameter reductiondegree decreases or is invariable with increasing compression rate.Furthermore, although a volume is used for calculating a compressionrate, for example, the volume of only a dark area having L* of 20 orsmaller may be used, or the compression rate may be calculated at asingle axis of L*.

Inverse Color-Space Converter 15

An inverse color-space converter 15 inversely converts a CIE L*a*b*color space into an RGB color space. Accordingly, a visibilityreproduction brightness image is generated, so that an image to beprinted by the image forming unit 208 as an example of an output devicemay be generated.

Next, the operation of the image processing apparatus 100 will bedescribed.

FIG. 11 is a flowchart illustrating an example of the operation of theimage processing apparatus 100 when generating a brightness reproductionimage.

As shown in FIG. 11, in step S100, the CPU 101 of the image processingapparatus 100 calculates an average brightness of a dark area when anoutput process to an output device is to be performed. Then, the processproceeds to step S101.

In step S101, the CPU 101 of the image processing apparatus 100calculates a reproduction parameter based on the average brightnesscalculated in step S100. Subsequently, the process proceeds to stepS102.

In step S102, the CPU 101 of the image processing apparatus 100generates a brightness reproduction image in accordance with Expression1 based on a brightness image to be output to the output device and thereproduction parameter calculated in step S101. Then, the process ends.

Second Exemplary Embodiment

Next, a second exemplary embodiment will be described.

The first exemplary embodiment described above relates to an examplewhere a reproduction parameter is estimated by using the averagebrightness of a dark area alone (see Expression 8). In the secondexemplary embodiment, a reproduction parameter is estimated in view ofthe percentage of the size (i.e., the number of pixels) of a dark area.

The following description focuses on differences from the firstexemplary embodiment described above, and redundant explanations will besimplified or omitted.

In order to take into account the percentage of the number of pixels ina dark area, the coefficient a in Expression 8 described in the firstexemplary embodiment is replaced with a coefficient a(r) serving as afunction of the percentage of the number of pixels in the dark area, asindicated in Expression 10.

α=a(r)×*_(ave)   Expression 10

For example, assuming that the coefficient a(r) in Expression 10 is alinear function, the coefficient a(r) may be expressed as in Expression11.

a(r)=α₀+α₁ ×r   Expression 11

Then, according to Expression 10, the reproduction parameter may beexpressed as in Expression 12.

α=(a ₀+α₁ ×r)×L* _(ave)   Expression 12

As indicated in Expression 12, the coefficient a in Expression 8 servesas the function of the percentage of the number of pixels in the darkarea, so that the reproduction parameter may be estimated in view of thepercentage of the number of pixels in the dark area. When this isexplained by using a drawing, the coefficient is fixed regardless of thepercentage of the size (i.e., the number of pixels) of the dark area inthe case of Expression 8, as shown in FIG. 7. In contrast, in the caseof Expression 11, the coefficient varies in accordance with thepercentage of the number of pixels in the dark area, as shown in FIG.12. Accordingly, the coefficient increases with increasing percentage ofthe number of pixels in the dark area, so that the reproductionparameter may be increased even when the average brightness of the darkarea is the same. Similar to the above, a₀ and a₁ may each be acoefficient set such that a reproduction parameter set based on, forexample, a sensory evaluation test executed in advance has a minimalerror in accordance with the least squares method.

Although assumed as being a linear function as an example, thecoefficient a(r) may be a multidimensional function such that therelationship that the coefficient has with the percentage of the numberof pixels in the dark area becomes nonlinear, as shown in FIG. 13,whereby the reproduction parameter may be estimated with higheraccuracy.

In detail, assuming that the coefficient a(r) is a multidimensionalfunction, the coefficient a(r) may be expressed as in Expression 13.

$\begin{matrix}{{a(r)} = {\sum\limits_{i = 0}^{n}( {a_{i} \times r^{i}} )}} & {{Expression}\mspace{14mu} 13}\end{matrix}$

In the above examples, a reproduction parameter is calculated bycalculating a feature value based on a threshold value at a certainlightness hierarchical level. Alternatively, as indicated in Expression14, a reproduction parameter may be set by calculating feature values atmultiple lightness hierarchical levels.

$\begin{matrix}{\alpha = {\sum\limits_{i = 0}^{n}( {{a_{i}( r_{i} )} \times L_{i}*_{ave}} )}} & {{Expression}\mspace{14mu} 14}\end{matrix}$

For example, when n=1, the reproduction parameter may be expressed as inExpression 15 by expanding Expression 14.

α=α₀(r ₀)×L ₀*_(ave)+α₁(r ₁)×L ₁*_(ave)   Expression 15

A method for setting multiple lightness hierarchical levels will bedescribed with reference to FIG. 5 as an example. For example, thelightness at which the windows alone are extracted as low brightnessareas in FIG. 5 is defined as a first lightness hierarchical level. Animage binarized at this lightness hierarchical level is shown in FIG.14. A feature value calculated at this lightness hierarchical level isexpressed in the first term of the right hand side of Expression 15.Then, the lightness at which the windows and the trees are extracted aslow-to-intermediate brightness areas in FIG. 5 is defined as a secondlightness hierarchical level. An image binarized at this lightnesshierarchical level is shown in FIG. 15. A feature value calculated atthis lightness hierarchical level is expressed in the second term of theright hand side of Expression 15. Specifically, a reproduction parameteris set by using the average brightness for each lightness hierarchicallevel obtained by separating the dark areas into multiple lightnesshierarchical levels.

By using multiple lightness hierarchical levels in this manner, thefeature values at the multiple lightness hierarchical levels may betaken into account, so that the weight on each parameter may be varieddepending on the lightness hierarchical level, whereby a reproductionparameter with higher versatility may be estimated. The coefficientsrelated to a₀ and a₁ may each be set such that a reproduction parameterset based on, for example, a sensory evaluation test executed in advancehas a minimal error in accordance with the least squares method, or mayeach be set such that the weight increases with increasing lightnesshierarchical level of the dark area.

Third Exemplary Embodiment

Next, a third exemplary embodiment will be described.

In the first exemplary embodiment and the second exemplary embodimentdescribed above, the compression rate of the color gamut of the outputdevice relative to that of the input device is saved in advance asinformation, and the calculation is performed uniformly by using thecompression rate. Thus, the compression rate varies depending on thedevice, but does not vary depending on the image. In the third exemplaryembodiment, the compression rate is calculated based on pixelinformation of an input image, so that the compression rate variesdepending on the image. Accordingly, an output-device optimalreproduction parameter may be set more appropriately in accordance withthe image.

The following description focuses on differences from the firstexemplary embodiment and the second exemplary embodiment describedabove, and redundant explanations will be simplified or omitted.

FIG. 16 illustrates a block diagram expressing the third exemplaryembodiment of the disclosure. The color space converter 11, thereproduction parameter analyzer 12, the output-deviceoptimal-reproduction-parameter calculator 13, the visibility reproducer14, and the inverse color-space converter 15 are identical to those inthe first exemplary embodiment.

A color-gamut compression color space converter 21 converts aninput-device RGB image into a lightness/chromaticity color space, suchas a CIE L*a*b* color space, compressed to the color gamut of the outputdevice based on a preliminarily-saved profile of the output device. Thecomponents of CIE L*, a*, b* respectively indicate lightness,chromaticity in the red-green direction, and chromaticity in theblue-yellow direction. In the third exemplary embodiment, a* and b* willbe defined as chromaticity components, and L* will be defined as abrightness component. The color space may be of any type so long as thecolor space is separable into lightness and chromaticity components,such as Lαβ, CAM02, and HSV.

A color-gamut compression-rate calculator 22 compares the chromaticityand brightness components in the input-device RGB image converted by thecolor space converter 11 with the chromaticity and brightness componentscompressed to the output-device color gamut converted by the color-gamutcompression color space converter 21 for each pixel, so as to calculatea compression rate. Although the calculation method is basically thesame as that in the first and second exemplary embodiments, the data tobe used for the calculation is input image information. Specifically,the compression rate of each pixel in the RGB image is calculated basedon a*, b*, and L* of all the pixels acquired by the color-gamutcompression color space converter 21 and a*, b*, and L* of all thepixels converted by the color space converter 11.

As an alternative to each of the above exemplary embodiments in whichthe image processing program is preliminarily stored (installed) in theROM 102 or the storage unit 104, the program may be provided by beingstored in a storage medium, such as a compact disk read-only memory(CD-ROM), a digital versatile read-only memory (DVD-ROM), or a universalserial bus (USB) memory. As another alternative, the program may bedownloaded from an external apparatus via a network.

The disclosure is not limited to the first to third exemplaryembodiments described above, and permits various modifications andapplications so long as they do not depart from the scope of thedisclosure.

In the exemplary embodiments above, the term “processor” refers tohardware in a broad sense. Examples of the processor include generalprocessors (e.g., CPU: Central Processing Unit) and dedicated processors(e.g., GPU: Graphics Processing Unit, ASIC: Application SpecificIntegrated Circuit, FPGA: Field Programmable Gate Array, andprogrammable logic device).

In the exemplary embodiments above, the term “processor” is broad enoughto encompass one processor or plural processors in collaboration whichare located physically apart from each other but may work cooperatively.The order of operations of the processor is not limited to one describedin the exemplary embodiments above, and may be changed.

The foregoing description of the exemplary embodiments of the presentdisclosure has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thedisclosure to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the disclosure and its practical applications, therebyenabling others skilled in the art to understand the disclosure forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of thedisclosure be defined by the following claims and their equivalents.

1. An image processing apparatus comprising: a processor configured to:adjust a reproduction parameter for correcting image visibility bychanging a degree by which the reproduction parameter is to be changedfrom a reference value in accordance with a ratio of a second colorgamut to a first color gamut, the first color gamut indicating a rangein which an image is expressible by a first device, the second colorgamut indicating a range in which an image is expressible by a seconddevice, wherein the reproduction parameter is derived by using anaverage brightness of a dark area in which a pixel value satisfies apredetermined criterion.
 2. (canceled)
 3. The image processing apparatusaccording to claim 1, wherein the reproduction parameter is derived byusing an average brightness of each lightness hierarchical levelobtained by separating the dark area into a plurality of lightnesshierarchical levels.
 4. The image processing apparatus according toclaim 1, wherein the reproduction parameter is derived from the averagebrightness and a size of the dark area.
 5. The image processingapparatus according to claim 3, wherein the reproduction parameter isderived from the average brightness and a size of the dark area.
 6. Theimage processing apparatus according to claim 4, wherein thereproduction parameter increases with increasing size of the dark area.7. The image processing apparatus according to claim 5, wherein thereproduction parameter increases with increasing size of the dark area.8. The image processing apparatus according to claim 1, wherein thedegree by which the reproduction parameter is to be changed from thereference value increases with increasing compression rate serving asthe ratio of the second color gamut to the first color gamut.
 9. Theimage processing apparatus according to claim 1, wherein the degree bywhich the reproduction parameter is to be changed from the referencevalue increases with increasing compression rate serving as the ratio ofthe second color gamut to the first color gamut.
 10. The imageprocessing apparatus according to claim 3, wherein the degree by whichthe reproduction parameter is to be changed from the reference valueincreases with increasing compression rate serving as the ratio of thesecond color gamut to the first color gamut.
 11. The image processingapparatus according to claim 4, wherein the degree by which thereproduction parameter is to be changed from the reference valueincreases with increasing compression rate serving as the ratio of thesecond color gamut to the first color gamut.
 12. The image processingapparatus according to claim 5, wherein the degree by which thereproduction parameter is to be changed from the reference valueincreases with increasing compression rate serving as the ratio of thesecond color gamut to the first color gamut.
 13. The image processingapparatus according to claim 6, wherein the degree by which thereproduction parameter is to be changed from the reference valueincreases with increasing compression rate serving as the ratio of thesecond color gamut to the first color gamut.
 14. The image processingapparatus according to claim 7, wherein the degree by which thereproduction parameter is to be changed from the reference valueincreases with increasing compression rate serving as the ratio of thesecond color gamut to the first color gamut.
 15. The image processingapparatus according to claim 1, wherein the processor is configured toadjust the reproduction parameter by changing the degree by which thereproduction parameter is to be changed from the reference value inaccordance with the ratio of the second color gamut to the first colorgamut, the first color gamut indicating a range in which a pixel of animage is expressible by the first device, the second color gamutindicating a range in which a pixel of an image is expressible by thesecond device.
 16. The image processing apparatus according to claim 1,wherein the processor is configured to adjust the reproduction parameterby changing the degree by which the reproduction parameter is to bechanged from the reference value in accordance with the ratio of thesecond color gamut to the first color gamut, the first color gamutindicating a range in which a pixel of an image is expressible by thefirst device, the second color gamut indicating a range in which a pixelof an image is expressible by the second device.
 17. The imageprocessing apparatus according to claim 3, wherein the processor isconfigured to adjust the reproduction parameter by changing the degreeby which the reproduction parameter is to be changed from the referencevalue in accordance with the ratio of the second color gamut to thefirst color gamut, the first color gamut indicating a range in which apixel of an image is expressible by the first device, the second colorgamut indicating a range in which a pixel of an image is expressible bythe second device.
 18. The image processing apparatus according to claim4, wherein the processor is configured to adjust the reproductionparameter by changing the degree by which the reproduction parameter isto be changed from the reference value in accordance with the ratio ofthe second color gamut to the first color gamut, the first color gamutindicating a range in which a pixel of an image is expressible by thefirst device, the second color gamut indicating a range in which a pixelof an image is expressible by the second device.
 19. The imageprocessing apparatus according to claim 5, wherein the processor isconfigured to adjust the reproduction parameter by changing the degreeby which the reproduction parameter is to be changed from the referencevalue in accordance with the ratio of the second color gamut to thefirst color gamut, the first color gamut indicating a range in which apixel of an image is expressible by the first device, the second colorgamut indicating a range in which a pixel of an image is expressible bythe second device.
 20. A non-transitory computer readable medium storinga program causing a computer to execute a process, the processcomprising: adjusting a reproduction parameter for correcting imagevisibility by changing a degree by which the reproduction parameter isto be changed from a reference value in accordance with a ratio of asecond color gamut to a first color gamut, the first color gamutindicating a range in which an image is expressible by a first device,the second color gamut indicating a range in which an image isexpressible by a second device, wherein the reproduction parameter isderived by using an average brightness of a dark area in which a pixelvalue satisfies a predetermined criterion.